Lesson 5 marks a pivotal shift in COMP5511. We are moving away from the deterministic logic of classical computing and the supervised certainty of early classification models into the probabilistic heart of Generative AI. In this session, we explore why modern AI models do not output "facts" but rather probability distributions, uncovering the mechanisms that allow LLMs to write poetry and diffusion models to paint from noise.

1. The Paradigm Shift: From Logic to Probability

• Beyond IF-THEN: Transitioning from rigid rules to fluid statistical likelihoods.
• The End of Certainty: Understanding why GenAI results are non-deterministic by design.
• Probability as a Tool: How the "Age of GenAI" treats uncertainty as a feature for creativity rather than a bug to be fixed.

2. The Probabilistic Engines of Creation

Generative AI relies on sampling from high-dimensional probability spaces. Whether generating text or images, the model navigates uncertainty to produce novelty:

• Large Language Models (LLMs): Predicting the next token not as a single choice, but as a distribution of possibilities.
• Diffusion Models: The art of reverse-engineering order from chaotic Gaussian noise.
• The Sampling Process: How randomness is harnessed to prevent repetitive and "robotic" outputs.

3. Agents in Unpredictable Worlds

Autonomous Agents must navigate "Open Worlds" where every action carries a degree of unknown risk and reward.